Classification Algorithm Sensitivity to Training Data with Non Representative Attribute Noise

We present an empirical comparison of major classification algorithms when training data contains attribute noise levels not representative of field data. Although conventional wisdom indicates that training data should contain noise representative of field data, it can be difficult to ensure representative noise levels. To study classification algorithm sensitivity, we develop an innovative experimental design using noise situation (under or over representation of training noise), algorithm, noise level, and training set size as factors. We consider situations of uniform attribute noise levels on all attributes, variable noise levels, and noise levels assigned by attribute importance. Our results contradict conventional wisdom indicating that investments to achieve representative noise levels may not be worthwhile. In general, over representative training noise should be avoided while under representative training noise is less of a concern. However, the interactions among algorithm, noise level, and training set size indicate that these general results may not apply to particular practice situations.